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Abstract —Powering cellular networks with renewable energy 
sources via energy harvesting (EH) has recently been proposed 
as a promising solution for green networking. However, with 
intermittent and random energy arrivals, it is challenging to 
provide satisfactory quality of service (QoS) in EH networks. 
To enjoy the greenness brought by EH while overcoming the 
instability of the renewable energy sources, hybrid energy supply 
(HES) networks that are powered by both EH and the electric 
grid have emerged as a new paradigm for green communications. 
In this paper, we will propose new design methodologies for HES 
green cellular networks with the help of Lyapunov optimization 
techniques. The network service cost, which addresses both the 
grid energy consumption and achievable QoS, is adopted as 
the performance metric, and it is optimized via base station 
assignment and power control (BAPC). Our main contribution 
is a low-complexity online algorithm to minimize the long-term 
average network service cost, namely, the Lyapunov optimization- 
based BAPC (LBAPC) algorithm. One main advantage of this 
algorithm is that the decisions depend only on the instantaneous 
side information without requiring distribution information of 
channels and EH processes. To determine the network operation, 
we only need to solve a deterministic per-time slot problem, for 
which an efficient inner-outer optimization algorithm is proposed. 
Moreover, the proposed algorithm is shown to be asymptotically 
optimal via rigorous analysis. Finally, sample simulation results 
are presented to verify the theoretical analysis as well as validate 
the effectiveness of the proposed algorithm. 

Index Terms —Green communications, energy harvesting, hy¬ 
brid energy supply, base station assignment, power control, QoS, 
Lyapunov optimization. 

I. Introduction 

T he continuous growth of wireless applications combined 
with the proliferation of smart mobile devices has re¬ 
sulted in an unprecedented growth of wireless data traffic, 
which has contributed to a dramatic increase in the energy 
consumption and carbon emission of the Information and 
Communication Technology (ICT) sector. It is estimated that 
the annual carbon emissions and electric power consumption 
of the ICT industry will reach up to 235 Mto H] and 414 
TWh El, respectively, in 2020. Heterogeneous and small cell 
networks (HetSNets) provide an energy-efficient paradigm to 
improve the network capacity, and thus have been regarded 
as one of the most promising solutions for realizing green 
radio a, a. However, with the dense deployment of base 
stations (BSs) in HetSNets, the overall energy consumption 
and carbon footprint will still be high 0. Consequently, it is 
urgent to seek alternative green energy sources for wireless 
networks. 
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The recent advances of energy harvesting (EH) technologies 
enable the BSs with EH components to capture ambient 
recyclable energy, e.g., solar radiation and wind energy, which 
is promising to achieve green networking 0,171. By introduc¬ 
ing EH capabilities to the next-generation cellular networks, 
potentially 20% of their CO 2 emission can be reduced fSl . 
Nevertheless, since the surrounding harvestable energy de¬ 
pends highly on environmental factors such as location and 
weather, the harvested energy is unstable by nature. As a 
result, it is challenging to maintain satisfactory quality of 
service (QoS) if communication nodes are solely powered by 
the harvested renewable energy. To enjoy the environmental 
friendliness of EH, and also to overcome the unreliability 
of the renewable energy sources, wireless networks with a 
hybrid energy supply (HES), where EH and the electric 
grid coexist, will be an ideal solution. While HES networks 
have attracted recent attention, they also bring new design 
challenges. In particular, communication protocols developed 
either for conventional grid-powered cellular networks or EH 
systems cannot take the full benehts of the heterogeneous 
energy sources in HES networks. In this paper, we shall 
propose new design methodologies for HES wireless networks, 
which will provide valuable guidelines for developing green 
cellular networks supported by renewable energy sources in 
the near future. 

A. Related Works and Motivations 

EH communications have attracted signihcant attention 
from academia in recent years. It was revealed that, with either 
the save-then-transmit protocol or the best-effort protocol, 
the capacity of the point-to-point additive white Gaussian 
noise (AWGN) channel can be achieved if the transmitter is 
powered by EH ||9l. This result indicates the benehts of EH 
communications from the information theoretical perspective. 
However, as the harvested energy is intermittent and sporadic, 
on one hand, energy management in EH systems should be 
based on the channel side information (CSI) as in conventional 
systems, but on the other hand, it should be adaptive to the en¬ 
ergy side information (ESI). With non-causal side information 
(SIU including the CSI and ESI, the maximum throughput 
of point-to-point EH fading channels can be achieved by the 
directional water-filling (DWE) algorithm Qo). The study was 
later extended to broadcast channels m, multiple access 
channels (TJ), and cooperative communications systems ifTSl , 

* ‘Causal Sr refers to the case that, at any time instant, only the past and 
current SI is known, while ‘non-causal SF means that the future SI is also 
available. 
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113. Besides these, scenarios with more practical assumptions 
on SI have been investigated in Qsi-iini- 

Transmission protocols for HES systems have also been 
recently studied, where the main focus is on point-to-point 
systems. Given the grid energy budget, a two-stage DWF 
algorithm was proposed in m to achieve the optimal through¬ 
put with non-causal SI at the transmitter. A similar problem 
was investigated in im, where a low-complexity recursive 
geometric water-filling algorithm was derived. In ll20ll . power 
allocation strategies for weighted energy cost minimization in 
a point-to-point HES link were proposed. Eor wireless links 
with hybrid energy sources, optimal power allocation policies 
to minimize the non-harvested energy consumption with delay- 
constrained data traffic requirement were proposed in ED. 
And resource allocation policies to maximize the energy 
efficiency in HES OFDMA systems were developed in Il22l . 
To realize green networking, more recently, powering cellular 
networks with hybrid energy supplies has been proposed 0, 
(ED-ES- The design in the network setting becomes more 
challenging since more decisions should be made, and more 
SI will be needed. To save the grid energy consumption, the 
green energy utilization optimization problem was solved in 
ll^ assuming full ESI was available, and a green energy 
and latency-aware user association scheme was proposed in 
ll24l . In ll25l . a sleep control scheme for HES networks was 
developed, while joint energy cooperation and communication 
cooperation for HES coordinated multi-point (CoMP) systems 
was proposed in ll26l . 

To simplify the design, previous studies on HES networks 
either ignore the accumulation of harvested energy at BSs 
ll24l . 123, or assume non-causal ESI is available (ED, ESl . 
which cannot fully capture the intermittency and randomness 
of EH. In general, for more practical online scenarios with 
causal SI, the optimal transmission policies remain unknown. 
For many online cases, the design problem can be formulated 
as a Markov Decision Process (MDP) problem, and thus can 
be solved in principle. However, due to the huge dimension 
of system states in HES networks, the complexity of the 
MDP solutions is unacceptable. Although heuristic policies 
can be developed, they generally do not have any performance 
guarantees. Motivated by these limitations in existing works, 
in this paper, we will investigate how to design practical 
online transmission protocols for HES cellular networks with 
desirable properties such as low complexity and theoretical 
performance guarantees. Specifically, Lyapunov optimization 
will be used as the main tool. Such techniques have a long 
history in the field of discrete stochastic processes and Markov 
chains ED- Moreover, it has been one of the most important 
methods for delay-aware resource control problems in wireless 
systems |E11, while application in EH networks was first 
proposed by Huang et al. 03. The algorithms developed from 
the Lyapunov optimization techniques enjoy various attractive 
properties, e.g., little requirement of prior knowledge, low 
computational complexity, and quantifiable worst-case perfor¬ 
mance, which make them a good fit for HES networks. 


B. Contributions 

In this paper, we will develop effective online algorithms to 
optimize green cellular networks powered by hybrid energy 
sources based on Lyapunov optimization techniques. Our 
major contributions are summarized as follows: 

• We consider a multi-user HES cellular network, and 
a network service cost that incorporates both the grid 
energy consumption and achievable QoS is adopted as 
the performance metric. The network service cost mini¬ 
mization (NSCM) problem, which is an intractable high- 
dimension Markov decision problem, is first formulated 
assuming causal SI. A modified NSCM problem with 
tightened battery output power constraints is then pro¬ 
posed, which will assist the algorithm design based on 
Lyapunov optimization techniques. 

• A low-complexity online Lyapunov optimization-based 
base station assignment and power control (LBAPC) 
algorithm is proposed for the modified NSCM problem, 
which also provides a feasible solution to the original 
problem. In each time slot, the network operation only 
depends on the optimal solution of a deterministic opti¬ 
mization problem, which can be solved efficiently by a 
proposed inner-outer optimization algorithm. 

• Performance analysis for the LBAPC algorithm is con¬ 
ducted. It is shown that the proposed algorithm can 
achieve asymptotically optimal performance of the origi¬ 
nal NSCM problem by tuning a set of control parameters. 
Moreover, it does not require statistical information of the 
involved stochastic processes including both the channel 
and EH processes, which makes it also applicable in 
unpredictable environments. 

• Simulation results are provided to verify the theoreti¬ 
cal analysis, especially the asymptotic optimality of the 
LBAPC algorithm. Moreover, the effectiveness of the 
proposed policy is demonstrated by comparison with a 
greedy transmission scheme. It will be shown that the 
LBAPC algorithm not only achieves significant perfor¬ 
mance improvement in terms of the network service cost, 
but also greatly reduces both the network grid energy 
consumption as well as the packet drop ratio. Moreover, 
it can more efficiently utilize the available spectrum. 

The organization of this paper is as follows. In Section II, we 
introduce the system model. The NSCM problem is formulated 
in Section III. The LBAPC algorithm for the NSCM problem 
is proposed in Section IV and its performance analysis is 
conducted in Section V. We present the simulation results in 
Section VI and conclude this paper in Section VII. 

H. System model 

We consider a multi-user HES wireless network with an 
EH-BS (6i), an HES-BS (B 2 ), and K mobile users (MUs), 
as shown in Fig. 1. The index set of the MUs is denoted as 
K. = {1, • • • ,K}. Two BSs coordinate to serve the MUs. The 
EH-BS, which can be a small cell to increase system capacity 
Q, is equipped with an EH component and powered purely by 
the harvested renewable energy, with the maximum transmit 
power given by The HES-BS is not only mounted with 
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Fig. i. A multi-user HES wireless network with one EH-BS, one HES-BS 
and four mobile users. 


an EH component, but also connected to the electric grid, i.e., 
it can utilize both the harvested energy and the grid energy, 
with the maximum transmit power denoted as pg This HES- 
BS may serve as a macro-BS to guarantee coverage. Both 
and Pg“ are assumed to be bounded. In this paper, the HES- 
BS acts as the decision center and collects all the SI needed 
for decision making. Eor ease of reference, we list the key 
notations of our system model in TABLE I. 

Time is slotted, and denote r as the time slot length and 
T = {O,!,-- } as the set of time slot indices. Downlink 
transmission is considered. Particularly, we assume that at the 
beginning of each time slot, a packet with R bits arrives from 
upper layers at the BS side for each MU. These packets may 
be transmitted via either the EH-BS or the HES-BS in the 
following time slot. It may also happen that neither of the BSs 
is able to deliver the packet. BS assignment and power control 
will be adopted to optimize the network. Denote Jj ^ G {0,1}, 
with j € {Si, 62 ,1?}, as the BS assignment indicator for MUfe 
in the fth time slot, where fe = 1 fe = 1 ) indicates 
that the EH-BS (HES-BS) is assigned to serve MUfc, and 
I\) ^ means neither of the BSs will transmit the packet, 

i.e., the packet is dropped. These indicators are subjected to 
the following operation constraint; 

^ 4, = i,vfc€/c,fer. (1) 

The transmit powers of the EH-BS and the HES-BS for 
MUfc in time slot t are denoted as p^^ and pg^ ^., re¬ 
spectively. As pg^ f. originates from the harvested energy 
at the EH-BS, for convenience it is also denoted as pj^^ f., 
while pg^ f. consists of both the harvested energy, denoted 
as j., and the grid energy, denoted as p{j j,, i.e., pg^ = 
p ^2 k Pg k- work, the energy consumed for purposes 

other than transmission, e.g., cooling and baseband signal 
processing, is neglected. 

The EH processes are modeled as successive energy packet 
arrivals, i.e., at the beginning of each time slot, energy 
packets with £[ and £2 arrive at the EH-BS and the HES- 
BS, respectively. We assume £{’& are independent and 

identically distributed (i.i.d.) among different time slots with 
the maximum value Although the i.i.d. EH 

model is idealized, it captures the intermittent nature of the 
EH processes, and thus it has been widely adopted in the 
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R 
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P*ki,k 
PG,k 
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ei (SI) 
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TABLE I 

Summary of Key Notations 

Description 

The EH-BS (HES-BS) 

Index set of the MUs 
Index set of the time slots 
Packet size 

BS assignment indicator for MU^ in time slot t 
Maximum transmit power at the EH-BS (HES-BS) 
Transmit power of the EH-BS (HES-BS) for 
MUfe in time slot t 

Power consumption of harvested energy at the 
EH-BS (HES-BS) for MU^ in time slot t 
Power consumption of grid energy at the 
HES-BS for MU/c in time slot t 
Battery energy level at the EH-BS (HES-BS) 
in time slot t 

Harvestable energy at the EH-BS (HES-BS) 
in time slot t 

Maximum value of {€2) 

Harvested energy at the EH-BS (HES-BS) 
in time slot t 

Channel gain from the EH-BS (HES-BS) 
to MUfc in time slot t 
Number of available orthogonal channels 
at the EH-BS (HES-BS) 

Cost incuiTed by per Joule of grid energy 
consumption (per packet drop) 

The weight of the grid energy cost (packet drop cost) 


literature, e.g., El, na, Eo). In each time slot, part of the 
arrived energy, denoted as e*, satisfying 

0<e‘ <£:j,jG{l,2},fGr, (2) 

will be harvested and stored in a battery, and it will be 
available for transmission from the next time slot. We start by 
assuming that the battery capacity is sufficiently large. Later 
we will show that by picking the values of e*’s, the battery 
energy levels are deterministically upper-bounded under the 
proposed algorithm, thus we only need finite-capacity batteries 
in the actual implementation. More importantly, including e^s 
as variables in the optimization facilitates the derivation and 
performance analysis of the proposed Lyapunov optimization- 
based algorithm, which will be elaborated in the following 
sections. Similar techniques were adopted in previous studies, 
such as ca and II 23 . Denote the battery energy levels of 
the EH-BS and the HES-BS at the beginning of time slot t 
as Bl and B^, respectively. Without loss of generality, we 
assume = 0 and Uj < +oo,j = 1, 2. Since the renewable 
energy that has not yet been harvested can not be utilized, the 
following energy causality constraint should be satisfied; 

PH^,kP 4 B* < -foo, j G (1,2}, Vf G T. (3) 

keK 

Thus, Bj evolves according to 

Pk,kP + J e { 1 , 2}, t G r. (4) 

keK 

The BSs will serve the MUs with multiple orthogonal 
channels, e.g., by adopting OEDMA as in the LTE standard 
ll^ . Static and orthogonal spectrum allocation is adopted for 
the two BSs, which is a popular scheme for tiered cellular 
networks ED, El- Specifically, we assume Nb^ + 
orthogonal channels with equal bandwidth w Hz are available. 
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where iVe^ > 1 of them are allocated for the EH-BS while the 
remaining > 1 channels are reserved for the HES-BS. In 
this paper, and are fixed and pre-determined. It is 
worthwhile to note that, and Nb 2 can be further optimized 
with the assistance of orthogonal access spectrum allocation 
strategies based on various system parameters, such as the 
EH conditions at both BSs and the network traffic demand 
El, El- This is beyond the scope of this paper and will be 
handled in our future works. As a result, at each time slot, the 
EH-BS and HES-BS can serve at most and Nis^ MUs, 
respectively. So the following channel assignment constraint 
should be met; 

(5) 

fceK 

The channels are assumed to be flat fading within the 
considered bandwidth and the channel gains are i.i.d. among 
different time slots. We denote the channel gains from the 
EH-BS (HES-BS) to MUfc as ^ ^). For simplic¬ 

ity, we further assume the channel gains from the BSs to 
the MUs are statistically independent and identical. Thus, 
k ~ (x), j = 1, 2, Vfc G /C, where Fs. (x) denotes the 

cumulative distribution functions of hjg. f.. As is the case in 
most of the existing works on EH communications, error-free 
channel estimation and feedback are assumed. Thus, perfect 
CSI is available at the transmitters. Therefore, the results in 
this paper can serve as a design guideline and a performance 
upper bound for cases where only imperfect CSI is available. 
Consequently, if the EH-BS (or HES-BS) serves MUfe in 
time slot t, the throughput is given by r fc) 

^ {f^B 2 ,k’PB 2 ,k)^’ '^here r{h,p) = wrlog^ is the 

Shannon-Hartley formula and cr is the noise power at the 
receiver. 


HI. Problem formulation 

In this section, we will first introduce the performance met¬ 
ric, namely, the network service cost. Then the network service 
cost minimization (NSCM) problem will be formulated, and 
its unique technical challenges will be identified. 

A. The Network Service Cost Minimization Problem 

As mentioned in Section II, it may happen that some data 
packets can not be successfully delivered to the corresponding 
MUs. For instance, the EH-BS does not have enough harvested 
energy while the channel from the HES-BS is in deep fading, 
or all the available channels are occupied. In such circum¬ 
stances, neither of the BSs is capable of transmitting the data 
packet, which induces a packet drop cost </?£). In some real¬ 
time applications, this packet may indeed be dropped, while 
for other applications, this will increase the delay. On the other 
hand, due to the intermittent and sporadic nature of energy 
harvesting, the HES-BS will need to use the grid energy for 
transmission from time to time, which incurs a grid energy 
cost (pa per Joule. Minimizing the grid energy consumption 
and maximizing the provided QoS are two important design 
objectives for HES networks. Thus, it is desirable to minimize 


both types of costs in order to optimize the system. In this 
paper, we will adopt the service cost as the performance 
metric, which is the weighted sum of the grid energy cost and 
packet drop cost. This metric was introduced in El^ where 
it was shown to be capable of adjusting the tradeoff between 
the grid energy consumption and the achievable QoS in HES 
networks. We will use the network service cost (NSC) as the 
performance metric for the studied HES network, which is the 
total service cost for all MUs and defined for the fth time slot 
as 

NSC* = ^ {waPGPa.k'^ + WDPDlD,k) ^ (6) 

keic 

where the first term presents the grid energy cost, the second 
term stands for the packet drop cost, and wa and wd are 
the weights of the grid energy cost and packet drop cost, 
respectively. When wa ^ wd, the network is grid energy 
sensitive. On the other hand, when wd wa, the network 
places more emphasis on the successful packet delivery, i.e., 
addresses on QoS more. Without loss of generality, we assume 
Pg, Pd, wq and wd are positive and bounded. 

Remark 1: Since the channel gains from the EH-BS (HES- 
BS) to different MUs are assumed to be i.i.d. in this paper, in 
order to guarantee fairness, the weights of the packet drop 
cost and grid energy cost are set to be the same for all 
MUs in (|6]l, i.e., wa and wd- For cases where the channel 
gains from the BSs to the MUs are not statistically identical, 
the weights can be adjusted accordingly, and the proposed 
Lyapunov optimization approach still applies. 

Our design objective will be to minimize the long-term 
average network service cost, which addresses both the net¬ 
work grid energy consumption and the achievable QoS, i.e., 
the percentage of successfully transmitted data packets of the 
MUs. For each MU, the following QoS constraint should be 
satisfied; 


i^j,k’Pj,k) > (1 - ^u.fc) R,k€lC,tGT, 

]e{Bi.B2} 

(7) 

i.e., if it is decided to transmit the packet, the throughput 
should be greater than the packet size. Consequently, the 
NSCM problem can be formulated as 


VI ; min lim — 

I*,p*,e* T—^ 4-00 T 


T -1 




K 


{PGPG,kT + Polo^k) 


.k=l 


S.t. O - dS]),©,© 

0 < plk < llkPr^"^ Vj € {Si,62}, teT,k€lC 

( 8 ) 




(9) 


keIC 

Tt 


Ilk G {0,1}, Vj G {Hi, 02 ,79},f G r, fc € /C 


PHi,kjPH2,k^PG,k > 0,yt €T,k € K., 


( 10 ) 

( 11 ) 


where ipa — wqpg, Pd = wdPd, I* = 


{7si,fe}, {lo^k} 


, = 


{pK.fc}’ {PG,k} 


and e* = [e^je^]. Thus we need to determine the BS 
assignment indicators I*, the power allocation p*, and the 
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harvested energy at both BSs, i.e., e\ and e\. In V\, ([Sj 
indicates that there will be no power allocated if an MU is 
not being served. The peak transmit power constraint and the 
power non-negativity constraint are imposed in (|9]) and (fTTI) . 
respectively. Moreover, the zero-one indicator constraint for 
the BS assignment indicators is represented by (doll. 


B. Problem Analysis 


IV. Online BS Assignment and Power Control: The 
LBAPC ALGORITHM 

In this section, we will develop the LBAPC algorithm to 
solve VI. In order to take the advantage of Lyapunov opti¬ 
mization, we will first introduce a modified NSCM problem to 
assist the algorithm design. Then the LBAPC algorithm will 
be proposed for the modified problem, which also provides a 
feasible solution to VI. In Section V, we will show that this 
solution is asymptotically optimal for VI. 


In the considered HES network, the system state includes 
the battery energy levels, harvestable energy at each BS and 
the channel states, and the action is the energy harvesting 
as well as the BS assignment and power control. It can be 
checked easily that the allowable action set in each time slot 
only depends on the current system state. Also, the state 
transition is Markovian, which depends on the current system 
state and action, and is irrelevant with the state and action 
history. Besides, the objective function in VI is the long-term 
average network service cost. Therefore, V\ is an infinite- 
horizon Markov decision process (MDP) problem. In principle, 
the optimal solution of VI can be obtained by standard 
MDP algorithms, e.g., the value iteration algorithm and linear 
programming reformulation approach m. Nevertheless, in 
both algorithms, we need to use finite states to characterize 
the system for practical implementation. For example, when 
iL = 4, if we use M = 10 states to quantize the energy level at 
each BS, E = 5 states to describe the harvestable energy, and 
L = 10 states to represent the channel gain, overall, there will 
be = 2.5 x 10^^ possible system states. For the 

value iteration algorithm, it will take unacceptably long time 
to converge to the optimal value function, while for the linear 
programming reformulation, a large-scale linear programming 
(LP) problem with more than 2.5 x 10^^ variables has to 
be solved, which is practically infeasible. Apart from the 
computational complexity, the memory requirement for storing 
the optimal policies is also a big challenge. Thus, it is critical 
to develop alternative approaches to handle VI. 

In the next section, we will propose a Lyapunov 
optimization-based BS assignment and power control 
(LBAPC) algorithm to solve VI, which enjoys the following 
favorable properties: 

• The decision of the LBAPC algorithm within each time 
slot is of low complexity, and there is no memory 
requirement for storing the optimal policy. 

• The LBAPC algorithm has no prior information require¬ 
ment on the channel statistics or the distribution of the 
renewable energy processes. 

• The performance of the LBAPC algorithm is controlled 
by a triplet of control parameters. Theoretically, by 
adjusting these parameters, the proposed algorithm can 
behave arbitrarily close to the optimal performance of 
VI. 

• An upper bound of the required battery capacity is 
obtained, which shall provide guidelines for practical 
installation of the EH devices and storage units. 


A. The Modified NSCM Problem 

Due to the energy causality constraint Q, the system’s de¬ 
cisions are not independent among different time slots, which 
makes the design challenging. This is a common difficulty 
for the design of EH and HES communication systems. We 
find that by introducing a non-zero lower bound, e// , on the 
battery output power, such coupling effects can be eliminated 
and the network operations can be optimized by ignoring (O 
at each time slot. Thus, we first introduce a modified version 
of VI as follows: 


V2 : min lim 

I*,p*,e* T—f-t-oo E 


1 




K 


iPGPG,k'^ + PDlo^k) 


lk=l 


s.t. (P) - @,©,0 - (HU) 


( 12 ) 


k^K 


- „max 


and 0 < eHj < Pb^J = 


where = {0}1J 
1, 2. Compared to Vl, an additional constraint on the battery 
output power is imposed in (fT^ . i.e., it is not allowed to be 
within (0,eHj) ,j = 1,2. Hence, V2 is a tightened version 
of VI, and thus any feasible solution for V2 is also feasible 
for VI. Denote the optimal values of VI and V2 as NSC^,]^ 
and NSC^ 2 , respectively. The following proposition reveals 
the relationship between NSCpj^ and NSCp 2 , which will 
later help show the asymptotic optimality of the proposed 
algorithm. 

Proposition 1: The optimal value of V2 is greater than that 
of VI, but no worse than the optimal value of VI plus a posi¬ 
tive constant v [chi , £^ 2 ), NSCp;^ < NSCp 2 < NSCp;^-|- 
where v {ch^Eh.,) = - E^^irf)) Kfio + 

• PG and p = { 2 ^ - aefj\. 

Proof: See Appendix A. ■ 

In general, the upper bound of NSC ^2 Proposition [T] is 
not tight. However, as en^yj = 1,2 goes to zero, v (eui , ) 

diminishes, as shown in the following corollary. 

Corollary 1: By letting and €h 2 approach zero, 

NSCp 2 can be made arbitrarily close to NSCpj^, i.e., 
lim V =^- 

lim eH 2 T 


, 6/^2 —^0 
Proof: First, 


£^ 2^0 
= -boo and 


lim 77 

eHi —^0 

ing the two equations, we have 


(pG = 0- Meanwhile, since 
Fbi {x) = 1, by combin- 

K 


lim 

X—)- + oo 

lim E^ (p) = 1 , i.e., 

EHi-s-o ^ 

lim (1 — ( 77 )) = 0. Thus, the desired result is obtained. 
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Proposition 1 bounds the performance of V2 by that of 
VI, while Corollary [T] shows that the performance of both 
problems can be made arbitrarily close. Actually, Corollary [T] 
fits our intuition, since when ejj 0, V2 reduces to VI. 
Moreover, we see from Proposition 1 that the upper bound 
of NSCp 2 equals NSCp^ + (ejTi j £^ 2 )’ which is a linear 
function of eH 2 - As a result, it converges to NSCpj^ linearly 
with respect to . Nevertheless, the convergence speed with 
respect to chi depends on the channel statistics. 

B. The LBAPC Algorithm 

In this subsection, we will propose the LBAPC algorithm 
for V2 based on Lyapunov optimization techniques. It is worth 
mentioning that the conventional Lyapunov optimization tech¬ 
niques, where the decisions are i.i.d., can not be applied to V2 
directly. As mentioned in the last subsection, this is because 
of the temporal correlation of the battery energy levels which 
makes the system’s decisions time dependent. Fortunately, 
for the modified NSCM problem, the weighted perturbation 
method provides an effective solution to circumvent this issue 

Ea. 

To present the algorithm, we will first define the perturbation 
parameters and the virtual energy queues, which are two 
critical elements. 

Definition 1: The perturbation parameters 9i and 62 for 
the EH-BS and HES-BS are bounded constants satisfying 

0 i > + {VKpD + 1 {K ^ , 

(13) 

O 2 > P^rr + {VKpD + 1{K ^ , 

(14) 

respectively. In (fT3l l and (fT4li . !{•} denotes the indicator 
function and 0 < E < +00 is a control parameter in the 
LBAPC algorithm with unit • cost“^. Here, “cost” denotes 
the unit of the network service cost. 

Definition 2: The virtual energy queues B\ and B\ are 
defined as 

^=B]-e„j€{l,2}, (15) 

which are shifted versions of the actual battery energy levels. 
We denote = {B\,B\) for convenience. 

As will be elaborated later, the proposed algorithm mini¬ 
mizes the weighted sum of the virtual queue lengths and the 
network service cost in each time slot, which shall stabilize 
around the perturbed energy level 9j and meanwhile minimize 
the network service cost. The LBAPC algorithm is summa¬ 
rized in Algorithm 1. In each time slot, the network operations 
are determined by solving the per-time slot problem, which is 
parameterized by the current system state, including the CSI 
and B*, and with all constraints in V2 except Q. (A Lyapunov 
drift-plus-penalty function will be defined in Section V, and 
one of its upper bounds will be derived, which happens to be 
the objective function of the per-time slot problem. Also, we 
will show that ignoring (|3ll in the per-time slot problem will 
not affect the feasibility of the LBAPC algorithm for V2.) 
This confirms that the LBAPC algorithm does not require 
prior knowledge of the harvesting processes and the channel 


statistics, which makes it also applicable for unpredictable 
environments. We will discuss the solution of the per-time slot 
problem in the next subsection, and analyze the feasibility as 
well as the performance of the proposed algorithm in Section 
V. 


Algorithm 1 The LBAPC Algorithm 
1 : At the beginning of each time slot, obtain the virtual 
energy queue state Bj, harvestable energy and channel 
gains hgj,k’ ^ /C,j = 1,2. 

2 : Decide e*, I* and p* by solving the per-time slot problem, 
i.e.. 


sb; 4 - 1 : 


PHj,kT 




kGiC 


K 


V {TGPG,kT + Polh^k) 


k=l 


s.t. (HI),®,®,© - (dll). 


3 : Update the virtual energy queues according to © and 

(US. 

4 : Set t = t + 1. 


C. Solving the Per-Time Slot Problem 


In this subsection, we will develop the optimal solution to 
the per-time slot problem, which consists of two components: 
the optimal energy harvesting, i.e., to determine e*, as well 
as the optimal BS assignment and power control, i.e., to 
determine I* and pL We find the closed-form solution of the 
optimal energy harvesting decision, and propose an efficient 
inner-outer optimization (lOO) algorithm to obtain the optimal 
BS assignment and power control decision. The results ob¬ 
tained in this subsection are essential for feasibility verification 
and for performance analysis of the LBAPC algorithm in 
Section V. 

Optimal Energy Harvesting: It is straightforward to show 
that the optimal amount of harvested energy e** is obtained 
by solving the following LP problem: 

2 


min 

0 <e*<£:* 


Vb: 

Ct /. -J 

and its optimal solution is given by 




(16) 


ef = £*-l{Bl<0},j = l,2. (17) 

Optimal BS Assignment and Power Control: Once decou¬ 
pling e‘ from the objective function, we can then simplify the 
per-time slot problem into the following optimization problem 

Bbpac: 


2 

Bbapc : min PH,,kr 

K 

+ V ^ {PGPG.k'T + PDlo^k) 

k=l 


S.t. o,®,© - dm), 
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which is a difficult mixed integer nonlinear programming 
(MINLP) problem. For each MU, there are 3 choices of BS 
assignment, so there are in total 3^ possible combinations. 
Moreover, for a fixed I‘, a non-convex power control problem 
should be solved, which further complicates the solution. Thus, 
the complexity of solving Pbapc with exhaustive search is 
prohibitively high. 

We will develop an lOO algorithm to solve T^bapCj which 
reduces the search space and avoids solving the non-convex 
power control problem. In the lOO algorithm, the inner 
optimization finds the optimal j.}, j,} and 

{Pq for given ^.} and {p\j^ j.}, while the outer opti¬ 
mization determines the global optimal {/g* and j,}. 
Denote V. = {k G j. = 1} and thus, Ig^ ^ = 0, 

pU,k=PG,k = o,vfc G V = }c\n.)E 

1) The Inner Problem: Given {Jg^ ^.} and {p^^ j.}, the 
inner problem can be written as 


V 


( —+ ^PGPG,k'^ ~ ^‘PDlB2,k'j 


S.t. 


lB2,k < Nb2 


E 

ken'= 

0 ^ PB2,k < lB2,kPB2 


Jh 2 ,k^{ 0 ,l},k€n'^ (18) 


^B2,kPB2,k > ^B2,kPB2,kyk ^ TP 
PB 2 ,k S Pb 7 '^ } , PH 2 ,k G ^2 


E 

keH‘ 

P*H2,k^PG,k >0,kG'H‘^ 


^E 

ken‘ 


(19) 

( 20 ) 

( 21 ) 


where X* = 


and 


PB2,k 


= 12 


( 2 ^ - 1 ) a! 


^B2,k^PH2,k^PG,k 

/ig 2 is called the channel inversion power of the HES-BS to 
MUfe channel. Pin is obtained by plugging the given {/g^ j,} 
and j.} into the objective function of PbapCj utilizing 
the fact that the transmit power for MUfe should be greater than 
the channel inversion power, and eliminating {Ijj ^,} with ([T]i. 
We denote the optimal value of Pin as Jin iTL). However, Pin 
is still a combinatorial optimization problem and not easy to 
solve. Fortunately, we find that, without loss of optimality, the 
MUs with lower channel inversion powers should be assigned 
to the HES-BS with higher priority, as shown in the following 
proposition. 

Proposition 2: There exists an optimal solution for Pin 
satisfying 


PB2,ki ^ G iSi, fco G 5o, (22) 

where <Si = {fc G TL‘^\lB 2 ,k = 1} Sq = \ Si = {k G 

f. = 0}. In other words, the MUs served by the HES- 
BS have better channel conditions than the remaining MUs. 


Proof: Suppose for an optimal solution of Pin, 3fci G 
<Si, feo G So such that pg^ > pg^ we can always serve 
MUfcp instead of MU^^ with the channel and transmit power 
that are originally allocated for MU^j, which will not increase 
the value of the objective function. Hence, there is also an 
optimal solution for Pin satisfying (l 22 li . which ends the proof. 


Therefore, we may concentrate on the solutions that satisfy the 
property in Proposition |2l which means that only the optimal 
number of MUs that are assigned to the HES-BS, denoted as 
TO*, need to be identified. We denote [r] as the index of the 
MU in TP with the fih smallest pg^ and TLi (0 < I < 
\TL‘^\,TLo = 0) as the set of MUs in with the I smallest 

Pb 2 ,IiV Besides, N = inm{NB^, ITP'I}. 

Since Pin is parameterized by B^, we will investigate the 
solution of Pin in the following three disjoint cases; 1 ) > 

V(pG, 2.) B\ > Q and 3) 0 < —Pi < Vpa- The optimal 
solution to the inner problem for these three cases will be 
shown later in Corollary |2] [3 and @1 respectively. 

The following lemma reveals a useful property for the 
solution in case 1 ). 

Lemma 1: When — P| > V(pG, for an optimal solution to 
Pin, we have p^^^ ^ = 0,Vfc G TP, i.e., no harvested energy 
in the HES-BS will be consumed. 

Proof: Suppose for an optimal solution, 3k G TP such 
that p^^^ > 0, it is feasible to construct a new solution with 

PH 2 ,k = 0 and p^Qj, = pI; ,. +p^H 2 ,k^ where the value of the 
objective function will decrease by {^—B\ — V(pc^ p\^^ f.T > 
0. By contradiction, the result is obtained. ■ 

According to Lemma [T] the transmit power of the HES-BS 
comes from the electric grid, with weight VipG > 0. Thus, it 
is optimal to serve the MUs with the channel inversion power. 
Hence, the optimal solution to Pin is given by Corollary |2] 
Corollary 2: For case 1), i.e., —> V(pc, the opti¬ 
mal number of MUs assigned to the HES-BS is to* = 
max i ■ l{-4casei ^ 0}, where A easel ^ 

^^■Acasel 

{I,-- - , N}\J2l=i PB 2 ,lk] ^ PBr^PGP%,,[^T < Pd}- And 
the optimal solution to Pin is given by 

^B2,k = C } 

PH2,k = 0 ykGic. (23) 

PG,k = ^B2,kPB2,k 


For case 2), as the weight of the harvested energy consump¬ 
tion is non-positive, i.e., P| > 0 , we can use Jf,ke'H‘= P*H 2 k ~ 
to decrease the length of the virtual energy queue by 
serving as many MUs as possible. Thus, the optimal solution 
to Pin can be summarized in Corollary [3] 

Corollary 3: For case 2), i.e., P| > 0, the opti¬ 

mal number of MUs assigned to the HES-BS is to* = 
inax i ■ l{Aase 2 0}, where Aase 2 = {i G 

iGAca.se2 

{1, • • • , ^}| ELi PB 2 ,[k] - Pb^}- = 0’ which means 

that all MUs in TL‘^ are experiencing deep fading, the HES-BS 
is not able to provide service. Otherwise, the optimal solution 
to Pin is given by 


^B2,k — S Pm*} 


PH2,k — 


E 


PB2AB2,k 

PB2,k 


Psr,Vfc G 1C. 


(24) 


kG'H‘= 


PG,k — 0 


^We focus on the non-trivial cases with ^ 0, where 0 is the empty set. 


Denote the total channel inversion powers of the MUs 
assigned to the HES-BS as ps, i.e., ps = EfceH^ 





For case 3), the following result reveals the relationship among 
PS’ {PH2,k} and 

Lemma 2: When 0 < —given 

(E..e«<=4.fc>oBvfce/c 


PHoM — 


PG,k 



(25) 


Proof: See Appendix B. ■ 

Lemma |2] shows that the transmit power comes either from 
harvested energy or grid energy, i.e., it can not be a mixture 
of them. Inspired by this property, given the number of MUs 
served by the HES-BS i, we have ^ = \{k G 7f°},VA: G 
/C, and the optimal power allocation is determined by (|25]) . 
Thus, we provide the optimal solution to Vm in Corollary 01 
Corollary 4: For case 3), i.e., 0 < —i ?2 — the 

optimal number of MUs assigned to the HES-BS is m* = 

arg .^min^3 J2ken‘ {-^IPH^^k + '^PGPG,k) ^ 

where AaseS = {0}U{* e {1, • •' , ELi - 
The optimal j,} to 'Pin is given by 

^B 2 ,k = ^ G 1C, and the optimal {pH 2 ,k} and 

{Pq can be obtained by using (l25l l accordingly. 

Thus, the inner problem is solved by obtaining m* and 
the associated power allocation policy by Corollary 2, 3 or 
4 depending on the value of 

2) The Outer Problem: For the EH-BS, given TL, the 
optimal transmit power is given by 


0 ,^ 

„max 

t rt 

min{Efce-M 

where 4 ( 2 ^ - l) 
can be formulated as 



= 0 

7 ^ 0 , B* < 0 

(26) 

Thus, the outer problem 


Pout : min $ {H) + Jin (B) (27) 

1 -LGJ^u 

where 

$(B) = 

fo, B = 0 

1 - VfD\n\, 'Hf%,B\>Q 

I -Bf max{ X; Pbi k^^H^}T - U(pd|B|, B 7^ 0,B^ < 0 
f feew 

(28) 

and Tn = {1C, C IC\\]C,\ < P%,,k < Pb^- 

The global optimal B*, i.e., {Jg* ^.}, can be obtained via 
searching all subsets of and the associated {p*j{^ j.} can 
be determined by ( |26] |. Basically, the lOO algorithm performs 


‘ "B 2 , 


a reduced-size search, which eliminates part of the possible 
combinations of I‘’s that are not optimal. More importantly, 
for fixed I‘, closed-form expressions for power allocation 
are derived to avoid solving the non-convex power control 
problem. In the worst case, there are 2^ subsets of Bh, and 
for the inner problem, we have to search m* from 1 to K, i.e., 
the complexity is O (2^ • K). Such exponential complexity 
is inevitable in BS assignment problems which are typically 
NP-hard. However, with a reasonable number of MUs, such 
complexity is acceptable given the increasing computation 
power at BSs. Overall, the lOO algorithm brings benefits of 
accelerating the searching processes compared to exhaustive 
search, while maintaining the optimality. In practice, at each 
time slot, the decision center, i.e., the HES-BS, collects the SI, 
runs the lOO algorithm and notifies the EH-BS of its decision. 
Details of the lOO algorithm are summarized in Algorithm 2. 


Algorithm 2 The lOO Algorithm 
1 : Compute the channel inversion power pg ^ based on 
fei J “ 2, fc G /C. 

2 : Set B = B«, J* = 0, = 1, = 0, = 0, 

P%,k=P%,k=Ptk = ^^ VfcG/C. 

3: While B 7 ^ 0 do 

4: Arbitrarily pick B G B, set /g^ j, = l{fcGB} and 

obtain fe} by (l26]) . 

5: Based on the value of B 2 , obtain {/g^ j.}, 

{p^H 2 fe}’ {Pg fe} ’^be associated Jin (B) with either 
Corollary |2l ^ or 0] 

6 : If (B) 4- Jin (B) < J* do 

7 : J* = $ (B) + Jin (B). 

8 : Update I**, p** with I‘, p‘. 

9 : Endif 

10 : B = B\B. 

11 : Endwhile 


V. Performance Analysis 


One unique advantage of the proposed algorithm is that we 
can provide theoretical performance analysis and characterize 
its asymptotic optimality, which will be pursued in this section. 
We will first prove the feasibility of the LBAPC algorithm 
for B2, which will be followed by the optimality charac¬ 
terization. Our analysis is based on Lyapunov optimization 
theory, where the Lyapunov drift function is the key element. 
During the analysis, an auxiliary optimization problem B3 
will be introduced, which bridges the optimal performance of 
B2 and the performance achieved by the proposed algorithm. 
Together with Proposition [T] this will establish the asymptotic 
optimality of the LBAPC algorithm for Bl. 

We verify the feasibility of the LBAPC algorithm by show¬ 
ing that the battery energy level is confined within a given 
interval, as demonstrated in the following proposition. 

Proposition 3: Under the LBAPC algorithm, the battery 


energy level B* is confined within 
Proof: See Appendix C. 


0 , 0 , 


TBrnax 


,J = 1 , 2 . 
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Proposition [3] shows that the energy causality constraint Q 
will not be violated, which indicates that the LBAPC algorithm 
is feasible for 7^2. It also implies that the required battery 
capacity in the proposed algorithm is 0 j + = 1 , 2 . 

In other words, given the size of the available energy storage 
at the BSs, i.e., = lj2, we can determine the control 

parameter V = minjVi, ¥ 2 ^ as 


F, = {Kf,ny 


TTriiax ^max^\ ^ ^ 

Cbj-Eh. -Pbj 


1 ( -L jT'ma.x ^max ^ 


(29) 




Furthermore, the bounds of the battery energy levels are 
useful for deriving the main result on the performance of the 
proposed algorithm. 

Next we will proceed to show the asymptotic optimality of 
the LBAPC algorithm, for which we first define the Lyapunov 
function as 


i (®‘)=5 E (»; - oj)*=5 f (»!)"+ 

i=i 


■ (30) 


It is a sum of squares of the virtual queue lengths, i.e., the 
distance between the battery energy levels and the perturbed 
energy levels. Accordingly, the Lyapunov drift function can 
be written as 


A = E L - L 




(31) 


Moreover, the Lyapunov drift-plus-penalty function can be 
expressed as 


Ay (j5‘) = A (b*) 


-fFE 


{yCPy^kT + ‘fDlD,k) l-S* 


.k&K 


(32) 

In the following lemma, we derive an upper bound for 
Ay which will play a critical role throughout the 

analysis of the LBAPC algorithm. 

Lemma 3: For arbitrary feasible decision variables 
e*,I*,p‘ for V2, Ay is upper bounded by 


Av 


(b<) 


< E 


0=1 \ keK / 


^ {’fGPG,k''' + yDlD,k) 


B* 


keK 


(33) 


C, 


where C = i + 


Proof: By subtracting 9j on both sides of @, 

+ e‘ - yy j = 1, 2. 


(34) 


'^Note that to guarantee Vj > 0, j = 1, 2, the values of ,j = l,2 can 
not be ai'bitrarily small. 


Squaring both sides of ( l34l i and adding up the equalities with 
j = 1 , 2 , we have 

o o / \ 2 


=E 4 ‘ + «FE 


PH^,kT 




2 

=E 


i=i 


kGiC 




PH^,kT 


k^K 




k&IC 




2B‘ ( e* 


-E^ 


A^PHj,kT 

k^K / 


(35) 

Dividing both sides of dLSl) by 2, adding 

^EfcGiC (‘FGPG.fc'r +^s Well as taking the 
expectation conditioned on B*, we can obtain the desired 
result. ■ 

Notice that the upper bound of Ay derived in Lemma 

|3] coincides with the objective function of the per-time slot 
problerrH in the LBAPC algorithm. To facilitate the perfor¬ 
mance analysis, we define the following auxiliary problem B3; 


T-l 


7^3 : min lim — 

I*,p*,e* T—f+OO T 




K 


E ‘PGPG,k''' + PdId,, 


.k=l 


s.t. (HD,©,®,© - (dll), 


T-l 


lim -VE 

T-s-l-oo T ^ 


i =0 


E 

.fceK 


PH^,k'r-ej 


= 0 ,j = l, 2 . 

(36) 

In 7^3, the energy causality constraint Q in V2 is replaced 
by (l36l l. i.e., the average harvested energy consumption equals 
the average harvested energy. In the following lemma, we will 
show that Vi is a relaxation of V2. 

Lemma 4: Vi is a relaxation of V2, i.e., NSCp 3 < 
NSCp 2 > where NSCp 3 is the optimal value of Vi. 

Proof: For any feasible solution of V2, based on the 
battery dynamics, we have 


B 


t+i 


— Bj — yy P^Hj^k''' + ep7 — 1) 2,f — 0,1, • • • ,T — 1. 


k&K 


(37) 

Summing up both sides of the above T equalities, taking the 
expectation, dividing both sides by T and letting T go to 
infinity, we have 
1 . 


lim -E [Bfl = 


T-l 


lim —E \B^] — lim — E 

T^ + ooT r^4-no T 


T-H-oo T 


t=o 


E PHj.k^' ~ ' 


.fce/c 


(38) 


^We drop the constant C in the objective function of the per-time slot 
problem. 
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Since < +cx), we have lim = 0, i.e., (|36] | is 

^ T-S-+00 ^ 

satisfied. Hence, any feasible solution of V2 is also feasible 
to 7^3, which ends the proof. ■ 

Besides, we find that there exists a stationary and ran¬ 
domized policy ll34ll . where the decisions are i.i.d. among 
different time slots, that behaves arbitrarily close to the optimal 
solution of P3. Meanwhile, the difference between E [e*] 


and E 




is arbitrarily small. It can be stated 
matheiriatically in the following lemma, which will help show 
the asymptotic optimality of the LBAPC algorithm. 

Lemma 5: For an arbitrary 6 > 0, there exists a stationary 
and randomized policy 11 for Vi, which decides e™, I™ and 
p™, such that ([TJ, (O, (|5]l, (l7])- (fT2l) are met, and the following 
inequalities are satisfied; 


E 


Y. {PGPG,k^ + PDl*D,k) 

.k&K 


< nsc ;,3 -f (5, f e r, 


(39) 


E 


E tn tn 

PH,,kT-e^ 

.kelC 


< 0S,j = i,2,f e T, 


(40) 


where p is a scaling constant. 

Proof: The proof can be obtained by Theorem 4.5 in 1^ . 
which is omitted for brevity. ■ 

In Section IV, we bounded the optimal performance of 
the modified NSCM problem V2 with that of the original 
NSCM problem VI, while in Lemma 0] we showed the 
auxiliary problem 7^3 is a relaxation of V2. With the assistance 
of these results, next, we will provide the main result in 
the section, which shows the worst-case performance of the 
LBAPC algorithm. 

Theorem 1: The network service cost achieved by the 
LBAPC algorithm, denoted as NSClbapCj is upper bounded 
by 

NSClbapc < NSCp3 -f v (cffi, Cffa) + y- (41) 


Proof: See Appendix D. ■ 

Remark 2: Theorem [T] implies that the performance of the 
LBAPC algorithm is controlled by the triplet {eHn^H 2 ^^)- 
By letting V +oo,€Hj —> 0,j = 1,2, the cost upper 
bound can be made arbitrarily tight, that is, the proposed 
algorithm asymptotically achieves the optimal performance 
of the original design problem 7^1. Note that in each time 
slot, the computational complexity of the LBAPC algorithm 
comes from the lOO algorithm, which is O {K ■ 2^) in the 
worst case regardless of the choice of (ejjj, V). However, 

approaching the optimal performance of 7^1 comes at the 
expense of a higher battery capacity requirement and longer 
convergence time to the optimal performance. The reason is 
that, under the LBAPC algorithm, the battery energy levels 
will be stabilized at 9j. As decreases or V increases, 
0 j increases accordingly, i.e., the perturbed energy levels are 
higher, and it will take longer time to accumulate the harvested 
energy, which postpones the arrival of the network stability 
and therefore delays the convergence. Thus, by tuning the 
control parameters, we can achieve different tradeoffs between 


the system performance and the battery capacity/convergence 
time. 

In this paper, the studied HES network consists of two typ¬ 
ical types of renewable energy-powered BSs, and the LBAPC 
algorithm provides an effective methodology for designing 
such a network. It is worthwhile noting that the proposed 
Lyapunov optimization approach can be generalized to HES 
networks with multiple BSs. However, as the network size in¬ 
creases, the computational complexity of obtaining the optimal 
solution of the per-time slot problem increases accordingly. 
Hence, low-complexity algorithms with performance guaran¬ 
tees for the per-time slot problem deserve further investigation. 

VI. Simulation Results 

In this section, we will verify the theoretical results derived 
in Section V and evaluate the performance of the proposed 
LBAPC algorithm through simulations. We consider an HES 
wireless network with 4 MUs unless otherwise specified. In 
simulations, is uniformly distributed between 0 and 
with the average EH power given by Ph^ = Ef^^^ (2r)~^, the 
channel gains are exponentially distributed with mean god~^, 
where go = —40 dB is the path-loss constant and d = 50 m 
is the distance from the MUs to the BSs. In addition, r = 1 
ms, WG = 5, R = 2 Kbitfl w = 1 MHz, a = lO’^^ W, 
^ ^max = 1 w and Wb, = 7T = 4. The costs of the 
grid energy and packet drop are normalized, i.e., (pG = 1 
per Joule and ipu = 1 per packet drop. Eor comparison, we 
introduce the Cost-aware Greedy algorithm as a performance 
benchmark, which gives higher priority to using the harvested 
energy and optimizes the system cost at the current time slot. 
It works as follows: 

« Eirst, MUs will be assigned to the EH-BS one by one until 
the harvested energy is used up. The MUs with higher 
values of f, will have a higher priority to be served. 

• Second, the remaining MUs will be assigned to the HES- 
BS until the harvested energy is used up. The MUs with 
higher values of fig^ will have a higher priority to be 
served. 

• The remaining MUs will be assigned to the HES-BS one 
by one if the increment of grid energy cost is less than 
the decrement of the packet drop cost. Similarly, the MUs 
with higher fig^ ^ will be assigned first. 

A. Theoretical Results Verification 

In this subsection, we will verify the feasibility and asymp¬ 
totic optimality of the LBAPC algorithm developed in Propo¬ 
sition [ 3 ] and Theorem [H respectively. The values of 0i and 
02 are chosen as the values of the right hand side of (fT3]) 
and (O, respectively. To verify the feasibility, we show the 
battery energy levels in Pig. |2] First, we observe that the 
harvested energy keeps accumulating at the beginning, and 
finally stabilizes at the perturbed energy levels. The reason 
is that in the proposed algorithm the Lyapunov drift-plus- 
penalty function is minimized at each time slot. From the 

®If a packet is transmitted, the data rate of the link between its destined 
user and the assigned BS is R/t = 2 Mbps in the time slot right after the 
packet’s anival. 
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Fig. 2. Battery energy levels, Ph^ = Ph 2 — ^0 and wd = 0.01. 



Fig. 4. Network service cost and required battery capacity vs. V, Phi = 
Ph 2 — 30 mW and wj^) = 0.01. 


curves, with a larger value of V or a smaller value of eh , 
the stabilized energy levels become higher, which coincides 
with the definition of the perturbation parameters in (foT l and 
(Ell. Also, we see that the energy levels are confined within 
0,6j + , which verifies Proposition |3] and confirms 

that the energy causality constraint is not violated under the 
proposed algorithm. The evolution of network service cost 
with respect to time is shown in Fig. |3] We see that, a 
larger value of V or a smaller value of ejj leads to better 
long-term average performance. Nevertheless, the algorithm 
converges more slowly to the stable performance. Besides, if 
(effi 7 £H 2 7 y) ^6 tuned properly, the proposed algorithm will 
greatly outperform the Cost-aware Greedy policy. 



0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 


Time (minutes) 


Fig. 3. Network service cost vs. time, Phi = Ph 2 — 30 mW and wd = 
0 . 01 . 

The relationship between the network service cost/required 
battery capacity and V is shown in Fig. |4] From Fig. 0] 
(a), we see that the network service cost achieved by the 
proposed algorithm decreases inversely proportional to V, and 
eventually it converges to the optimal value of V2, which 
verifies Theorem [T] i.e., the asymptotic optimality. However, 
as shown in Fig. 0] (b), the required battery capacity grows 


linearly with V, which is because the value of 9j is linearly 
increasing with V. Thus, V should be chosen to balance 
the achievable performance, convergence time and required 
battery capacity. For instance, if batteries with 100 mJ capacity 
are available, we can choose V = l.Ox 10“^ J^-cost“^ for the 
LBAPC algorithm, and then 47% performance gain compared 
to the benchmark will be obtained. 


B. Performance Evaluation 

We will show the effectiveness of the proposed algorithm 
and demonstrate the impacts of various system parameters 
in this subsection. The relationship between the network 
service cost and the harvesting power at the EH-BS, i.e., 
Phi, is shown in Fig. We see that the network service 
cost achieved by either policy is non-increasing with Phi, 
which is in accordance with our intuition since consuming the 
harvested energy incurs no cost. Also, the LBAPC algorithm 
significantly reduces the network service cost compared to 
the greedy algorithm. Besides, the influence of the number 
of channels at the EH-BS is also revealed. With = 1, 
i.e., in the spectrum limited scenario, increasing Ph^ brings 
negligible benefit to the performance in the benchmark policy, 
since the MU with the best channel condition to the EH-BS 
is served and this consumes little harvested energy, i.e., < 15 
mW. On the other hand, in the proposed algorithm, the network 
service cost keeps decreasing with Ph^ because it minimizes 
the Lyapunov drift-plus-penalty function at each time slot and 
the MU being served by the EH-BS is not necessarily the 
one with the highest channel gain. By increasing from 1 
to 2, the network service cost for both algorithms is greatly 
reduced, but the Cost-aware Greedy algorithm experiences 
performance saturation when Ph^ > 40 mW. This shows 
that the proposed algorithm can not only reduce the system 
cost, but also better utilize the spectrum resource. Meanwhile, 
increasing from 2 to 4 has noticeable impact on the 
greedy algorithm while the benefit to the LBAPC algorithm is 
minor, which is because the EH-BS becomes energy limited 
in this region. 
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Fig. 5. Network service cost vs. Phi, Ph 2 — 30 mW, ehi = ^H 2 — '^O 
mW, Csi = Cb 2 = 150 mJ. 


The network service cost versus the harvesting power at 
the HES-BS, i.e., Ptfj, is shown in Fig. |6] Similarly, the 
cost performance decreases as Ph^ increases. Nevertheless, 
there is no performance saturation at high since with 
Nb 2 = the MUs can always be served by the HES- 
BS. From Fig. |6] we also see the linear relationship between 
the network service cost and Ph 2 ^ which is because the 
harvesting energy and grid energy are co-located at the HES- 
BS. Additionally, by comparing Fig. |5] and Fig. |6l we observe 
that the performance improvement by increasing Ph^ is more 
obvious than increasing Ph^^ which is due to the diversity 
gain obtained from independent channels from the two BSs. 
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benchmark remains upon 1.0%, i.e., the LBAPC algorithm has 
the potential to meet a higher QoS requirement. From Fig. |7] 
the proposed algorithm not only outperforms the Cost-aware 
Greedy algorithm in terms of the network service cost, but 
it is also more competent to suppress both the grid energy 
consumption and the packet drops. This indicates that, in HES 
wireless networks, the optimal energy management should 
balance the current and future performance, as well as fully 
utilize the available SI, and the LBAPC algorithm is such a 
promising solution. 


(a) (b) 



Wd 1U£> 


Fig. 7. Grid power consumption and packet drop ratio vs. wo-, Pfij = 30 
mW, eHj = 40 mW, Cg. = 150 mJ, j = 1, 2. The curves with no markers 
are with = 1, while the curves marked with and “o” are with 

Njs-^ = 2 and 4, respectively. 


Finally, we show the relationship between the network 
service cost and the number of MUs in Fig.0 We see that the 
network service cost achieved by either policy is increasing 
with K. This is because the network traffic demand grows 
as K increases. Due to the insufficient amount of harvested 
energy, the networks with more MUs are prone to consume 
more grid energy and drop more packets, both of which 
contribute to the increase of the network service cost. Besides, 
the effectiveness of the proposed policy is again validated, 
and the performance gain compared to the benchmark policy 
expands as K increases. This highlights the importance of 
optimal energy management and the benefits of the proposed 
algorithm for HES cellular networks, especially when the 
renewable energy resource is scarce, or the traffic load is 
heavy. 


Fig. 6. Network service cost vs. Ph 2 ^ = 30 mW, chi = ^H 2 — 
mW, = C'g 2 = 150 mJ. 

The grid energy consumption and packet drop ratio achieved 
by different algorithms are shown in Fig. |2](a) and Fig.|2](b), 
respectively. As wd increases, the grid energy consumption 
increases, meanwhile, the packet drop ratio decreases. Thus, 
by adjusting the weights of the grid energy cost and packet 
drop cost, different tradeoffs can be achieved. When wd 
is sufficiently large, the packet drop ratio achieved by the 
proposed scheme approaches zero while that achieved by the 


VH. Conclusions 

In this paper, we proposed new design methodologies for 
HES green cellular networks based on Lyapunov optimization 
techniques. The network service cost, which is comprised of 
the grid energy cost and the packet drop cost, was adopted 
as the performance metric, and a BS assignment and power 
control (BAPC) policy was then developed to optimize the net¬ 
work. A practical online Lyapunov optimization-based BAPC 
(LBAPC) algorithm was proposed, which requires little prior 
knowledge and enjoys low computational complexity. Perfor¬ 
mance analysis was conducted which revealed the asymp- 
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Fig. 8. Network service cost vs. K, Phi = Ph^ = 30 mW, 
e_ffj = Cifj = 40 mW, Cgj = Cgj = 150 mJ. 


totic optimality of the proposed algorithm. Simulation results 
showed that the proposed LBAPC algorithm significantly 
outperformed the greedy transmission scheme in terms of the 
network service cost, grid energy consumption, as well as the 
achievable QoS. Our results demonstrated the effectiveness 
of Lyapunov optimization techniques to overcome the curse 
of dimensionality in MDP solutions, which was the primary 
barrier in online transmission protocols design for HES green 
cellular networks. It will be interesting to extend the proposed 
algorithm to more general HES networks as well as incorpo¬ 
rate more realistic EH models and multi-antenna techniques 
into consideration, and investigate other design problems such 
as interference management and user scheduling. 

Appendix 

A. Proof of Proposition Q] 

Since P2 is a tightened version of VI, we have NSC^j^ < 
NSC^ 2 - The proof for the other side of the inequality can be 
obtained by constructing a feasible solution for V2 based on 
the optimal solution of VI: 

• If ^ ^ S (OjCr/j), then the harvested energy of the 

fceK 

EH-BS will not be used in the constructed solution, and 
the MUs assigned to the EH-BS will experience packet 
drop. 

• If ^ £ (Oje/Tj), in the constructed solution, the 

fce/c 

harvested energy from the HES-BS will be replaced by 
the grid energy. 

It is straightforward to verify that the constructed solution is 
feasible to V2. Thus, 

K 

NSC ;,2 < NSC;,! + err, r • pg + ^ Pfc • kfo 

k=i 

< NSC^i -b errsT • (pc + (1 - Po) Kfo, 

where pk is the probability that k MUs can be served by the 
EH-BS with total transmit power eHi - Because of the coupling 
among the transmit power and the available channels, an exact 


expression of po is difficult to obtain. However, we can obtain 
a lower bound on po by ignoring the coupling effects, i.e.. 




< R 


= (p). By substituting 


Po > n I 

fce/c 

the lower bound of po into (l42l) . the result is obtained. 


B. Proof of Lemma |2] 


When py, £ 


0 , 




VtfiG 


, suppose there is a solution with 


£ {fc £ = 1}, p^^ fc > 0- Due to (fT^ . with the 

solution in (1251 ) instead, the value of the objective function will 

decrease by -B^ max { X) Ph^ fc’ ^^2 }r-Vf>GPi:'r > 
kew^ 

0, i.e., p^^ ^ = 0,Vfc £ /C is optimal. Also, as V(pG > 0, it is 
optimal to transmit with the channel inversion power. 


When py £ 


-_^ 2^2 

V<PG 


7 crr2 


since 




> 0, either 


E Ph 2 fc = 0 or err 2 is optimal. When T, Pk k = 0’ 
feew” ’ kf^H‘ 


PG,k — PB2,k^B2,k- 
Stead, the value of the objective function will decrease by 

Vf>GPT.T - i-Bl) eH 2 T > 0, i.e., X) Pk fc = ^^2 is 

optimal. 

When py £ (crf 2 j Pb 2 ^^] ’ contradiction, the optimal 
solution should satisfy 


I EfceW=^'G.fc^.B2,fc — Ve ^42) 

\j 2 ken-PH 2 ,k^k,k = max{(l - X)pY,eH 2 } ’ 

where A £ [0,1). Suppose there is a solution with fc £ {/c £ 
fe = 1}’ Pg fc > u®-’ A > 0. By using (|25]) instead, 

i.e., A = 0, the value of the objective function will decrease 
by 

UpgApst - .B 2max{(l - X)pY,eH 2 }'r - (“^2) Pst 
> V(pG^PY.T - B\{1- X) pyt - (-^2) PET 

= (^<PG - (“^ 2 )) PE At > 0, 


i.e., A = 0 is optimal. Similarly, as —Bl > 0, it is optimal 
to transmit with the channel inversion power. To summarize, 
(I 25 I) is optimal and thus Lemma |2] is obtained. 

C. Proof of Proposition |5] 

We shall first prove that B* is upper bounded by 9j -f 
= 1, 2, based on the optimal harvested energy in (fTTI) . 
Suppose 9j < Bj < 9j + and since e** = 0, we have 

< Bj < 9j + Otherwise, if 0 < < 9j, since 

ef = E], we have 5*+^ < B] + ef < 9^ + 

Next, we reveal an important property of the optimal 
solution of the per-time slot problem to assist the proof for 
the lower bound. It indicates that if the battery energy level at 
a BS is below a threshold, no harvested energy in this BS will 
participate in transmission, as shown in the following lemma. 

Lemma 6: If < p^^^r, then ^ = 0,j = l,2,fc £ 
K., where is the optimal battery output power obtained 

in the per-time slot problem. 
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Proof: We start with the single user case and omit the 

T, there is an optimal 


user index. Suppose when B* < Pb, 


solution where > 


0 Hk = 




— P — 

— Id — 


0). Since 

j - j - 'J—j 

Pfj- G rij, thus Pd. > f-Hj ■ With this solution, the value of the 
objective function in the per-time slot problem is no less than 
—BjeHjT > V(pD, which is greater than what is achieved by 
the solution with If) = Pkj — P^H^-j = Pg ~ P^h ~ 

0. For the multi-user scenario, again, suppose when Bj < 
p'§^^T, there is an optimal solution where fc > 0’ 

i-®-’ Thk^KPkj k — With this solution, the minimum 
value of the objective function is —B^h^t — E 


which is achieved when B^_j 


— 0 ?,-: 


max „max _ 

meanwhile 


3-J -r 

all the MUs are served with the harvested energy, and at least 
one of them is served by BS B-i-j. With the definition of 6 j, 


we are able to show —B^en T — E 


max 
t 


„max _ 
PSs-i^ 


> KV<pd, 


which is achieved by the solution with If f. = 1, V/c G /C, and 

J2kGJcP^lj,k — 0’ i-®-’ P*Hj,k = 0,Vfc G /C. ■ 

Based on the property of p*f^ ^ derived in Lemma |6] now 
we proceed to show B* is lower bounded by zero. Suppose 
0 < Bj < p'^f^T, according to Lemma |6] pf. ^ = 0,Vfc G /C, 

r, with (|9ll, 

>B]- Y. PH,,k^ > 0. As a 

keK 


B] 


thus > 

E t* ^ m 

PHi,k ^ Pb 

keK 


> 0. Otherwise, if B* > pg' 


thus B 


t+i 


result, Bj G 


0,0, 


rmax 


D. Proof of Theorem Q] 

Since the LBAPC algorithm obtains the optimal solution of 
the per-time slot problem, 

Av (b*) 


< E 


,P%,kT 


< E 


3 = 1 \ k&K / 

+ VJ2 {PGPG,k'^ + PDlk,k) \B 

eK 

3 = 1 \ k&K / 

+ ^ ^ + PDlolk) \B 

keK 

^B] (ef^ - '^P%,k'r\ \B 

=1 \ fceK / 


+ C 


= E 


3 = 1 


+ C 


-f LE 


(t) 


.keK 


C 


<gSj2 max{0j , -f V (NSCf^ + S) + C, 

3 = 1 

(44) 

where (|) is due to Lemma |5] By letting <5 go to zero, we 
obtain 

Ay (b‘) < LNSC;,3 -f C. (45) 

Taking the expectation on both sides of (l45T l. summing up 
the equations for t = 0, • ■ • T — 1, dividing by T and letting 


T -fcx), we have NSClbapc < NSCjjg -f A. By further 
utilizing Proposition [T] and Lemma |4] the theorem is proved. 
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